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Amendments to the Claims : 

This listing of claims replaces all prior versions and listings of claims in the application: 

Listing of Claims : 
1.-57. (Cancelled) 

58. (Currently Amended) A system comprising: 

one or more mobile devices on e or mor e comput e rs ; and 

a computer-readable medium coupled to the one or more mobile device one or mor e 
comput e rs having instructions stored thereon which, when executed by the one or mor e 
computers , cause a single one of the mobile devices cause the on e or more computers to perform 
operations comprising: 

providing a multi-modal user interface on a mobil e devic e, the multi-modal 
interface for assisting a warehouse worker in performing inventory management tasks in a 
warehouse that stores different items for sale in multiple storage locations disposed throughout 
the warehouse e nvironm e nt , and the multi-modal interface enabling output and user input of 
information using both first and second modalities that collectively include a voice-based 
modality and a touch-based modality, 

receiving a first user input identifying a job type mode, the first user input being 
input bv the warehouse worker using either the first modality or the second modality as chosen 
by the warehouse worker a- u se r of th e mobil e d e vice , and the job type mode being selected from 
among a stocking mode in which the user warehouse worker enters one or more items into a 
particular one of the multiple storage locations disposed throughout the warehouse a bin in th e 
warehouse environment , a picking mode in which the user warehouse worker removes the one or 
more items from the particular storage location t h o b in i n th e war e house environment , and a 
counting mode in which th e use r warehouse worker counts the one or more items in the 
particular storage location bin in th e war e hous e e nvironm e nt , 
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transmitting information identifying the user-selected job type mode to an 
inventory management system, in a single format which is consistent with the modality in which 
the first user input was received, 

responsive to transmitting the information, receiving a response to the information 
identifying the user-selected job type mode from the inventory management system, in two or 
more formats which are consistent with both the first modality and the second modality, the 
response identifying a location at which the user warehouse worker will perform a function 
related to the user-selected job type mode on the one or more items, a description of the one or 
more items, and a best route from a present location of the warehouse worker to the particular 
storage location us e r to th e bin , 

providing the response to th e us e r the warehouse worker using both audio and 
display components of the multi-modal user interface, in the two or more formats which are 
consistent with both the first modality and the second modality, 

receiving a second user input using either the first modality or the second 
modality as chosen by the warehouse worker th e us e r , the second user input indicating that the 
user warehouse worker has completed the selected function, and including count information or 
location information associated with the one or more items as observed by the warehouse worker 
th e us e r and, when the voice-based modality is used, a phonetically distinct checkword 
associat e d with the bin to indicate associated with and displayed at the particular storage 
location, the phonetically distinct checkword indicating completion of the selected function, 

updating inventory data for the one or more items based on the count information 
or the location information, the updated inventory data being accessible using the two or more 
formats which are consistent with both the first modality and the second modality, 

determining that the particular storage location is expected to be empty, 

responsive to determining that the particular storage location is expected to be 
empty, prompting the warehouse worker to confirm that the particular storage location is 
observed to actually be empty, using both the first modality and the second modality, and 

transmitting information indicating that th e user warehouse worker has completed 
the selected function related to the user-selected job type mode to the inventory management 
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system, in a single format consistent with the modality in which the second user input was 
received. 

59. (Previously Presented) The system of claim 58, wherein the operations further comprise: 
outputting the updated inventory data in a format consistent with either the first modality 

or the second modality, such that the inventory data is maintained during performance of the 
inventory management tasks. 

60. (Previously Presented) The system of claim 58, wherein the job type mode comprises job 
data. 

61 . (Previously Presented) The system of claim 58, wherein the multi-modal interface 
further enables output and user input information using a Radio Frequency Identification Device 
(RFID) modality, or a bar code modality. 

62. (Previously Presented) The system of claim 58, wherein the operations further comprise 
receiving a pick list identifying the one or more items and the function. 

63. (Previously Presented) The system of claim 62, wherein the operations further comprise: 
displaying the pick list using a visual user interface, and 

outputting, using voice synthesis, information identifying less than all items on the pick 

list. 

64. (Previously Presented) The system of claim 58, wherein the formats consistent with both 
the first modality and the second modality comprise HTML and VXML, respectively. 

65. -68. (Cancelled) 

69. (Previously Presented) The system of claim 58, wherein the first user input is received 
using a different modality than the second user input. 
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70. (Previously Presented) The system of claim 69, wherein: 

the first user input is received using the voice-based modality, and 
the second user input is received using the touch-based modality. 

71 . (Currently Amended) A computer-readable medium encoded with a computer program 
comprising instructions that, when executed, operate to cause a single mobile device comput e r to 
perform operations comprising: 

providing a multi-modal user interface on a mobile d e vic e, the multi-modal interface for 
assisting a warehouse worker in p erforming inventory management tasks in a warehouse that 
stores different items for sale in multiple storage locations disposed throughout the warehouse 
e nvironment , and the multi-modal interface enabling output and user input of information using 
both first and second modalities that collectively include a voice-based modality and a touch- 
based modality; 

receiving a first user input identifying a job type mode, the first user input being input by 
the warehouse worker using either the first modality or the second modality as chosen by the 
warehouse worker a user of th e mobil e devic e, and the job type mode being selected from among 
a stocking mode in which the use* warehouse worker enters one or more items into a particular 
one of the multiple storage locations disposed throughout the warehouse a bin in th e war e house 
environm e nt , a picking mode in which the use? warehouse worker removes the one or more 
items from the particular storage location th e bin in th e war e house environment , and a counting 
mode in which th e us e r warehouse worker counts the one or more items in the particular storage 
location bin in the warehouse environm e nt ; 

transmitting information identifying the user-selected job type mode to an inventory 
management system, in a single format which is consistent with the modality in which the first 
user input was received; 

responsive to transmitting the information, receiving a response to the information 
identifying the user-selected job type mode from the inventory management system, in two or 
more formats which are consistent with both the first modality and the second modality, the 
response identifying a location at which t h e us e r warehouse worker will perform a function 
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related to the user-selected job type mode on the one or more items, a description of the one or 
more items, and a best route from a present location of the warehouse worker to the particular 
storage location user to the bin ; 

providing the response to th e user the warehouse worker using both audio and display 
components of the multi-modal user interface, in the two or more formats which are consistent 
with both the first modality and the second modality; 

receiving a second user input using either the first modality or the second modality as 
chosen by the warehouse worker the user , the second user input indicating that the us e r 
warehouse worker has completed the selected function, and including count information or 
location information associated with the one or more items as observed by the warehouse worker 
th e- us e r and, when the voice-based modality is used, a phonetically distinct checkword 
associat e d with -th e bin to indicat e associated with and displayed at the particular storage 
location, the phonetically distinct checkword indicating completion of the selected function; 

updating inventory data for the one or more items based on the count information or the 
location information, the updated inventory data being accessible using the two or more formats 
which are consistent with both the first modality and the second modality; 

determining that the particular storage location is expected to be empty; 

responsive to determining that the particular storage location is expected to be empty, 
prompting the warehouse worker to confirm that the particular storage location is observed to 
actually be empty, using both the first modality and the second modality; and 

transmitting information indicating that the user warehouse worker has completed the 
selected function related to the user-selected job type mode to the inventory management system, 
in a single format consistent with the modality in which the second user input was received. 

72. (Previously Presented) The computer-readable medium of claim 71 , wherein the 
operations further comprise: 

outputting the updated inventory data in a format consistent with either the first modality 
or the second modality, such that the inventory data is maintained during performance of the 
inventory management tasks. 
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73. (Previously Presented) The computer-readable medium of claim 71, wherein the job type 
mode comprises job data. 

74. (Previously Presented) The computer-readable medium of claim 71, wherein the multi- 
modal interface further enables output and user input information using a Radio Frequency 
Identification Device (RFID) modality, or a bar code modality. 

75. (Previously Presented) The computer-readable medium of claim 7 1 , wherein the 
operations farther comprise receiving a pick list identifying the one or more items and the 
function. 

76. (Previously Presented) The computer-readable medium of claim 75, wherein the 
operations further comprise: 

displaying the pick list using a visual user interface, and 

outputting. using voice synthesis, information identifying less than all items on the pick 

list. 

77. (Previously Presented) The computer-readable medium of claim 71, wherein the formats 
consistent with both the first modality and the second modality comprise HTML and VXML, 
respectively. 

78. -81. (Cancelled) 

82. (Previously Presented) The computer-readable medium of claim 71 , wherein the first 
user input is received using a different modality than the second user input. 



83. 



(Previously Presented) The computer-readable medium of claim 82, wherein: 
the first user input is received using the voice-based modality, and 
the second user input is received using the touch-based modality. 
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84. (Currently Amended) A computer-implemented method comprising: 

providing a multi-modal user interface on a single mobile device, the multi-modal 
interface for assisting a warehouse worker in p erforming inventory management tasks in a 
warehouse that stores different items for sale in multiple storage locations disposed throughout 
the warehouse environment , and the multi-modal interface enabling output and user input of 
information using both first and second modalities that collectively include a voice-based 
modality and a touch-based modality; 

receivin g, by the mobile device, a first user input identifying a job type mode, the first 
user input being input by the warehouse worker using cither the first modality or the second 
modality as chosen by the warehouse worker a us e r of the mobile device , and the job type mode 
being selected from among a stocking mode in which the use* warehouse worker enters one or 
more items into a particular one of the multiple storage locations disposed throughout the 
warehouse a bin in th e war e h o us e e nvironm e nt , a picking mode in which the user warehouse 
worker removes the one or more items from the particular storage location th e bin in th e 
warehou se envi ronment, and a counting mode in which th e us e r warehouse worker counts the 
one or more items in the particular storage location bin in the warehouse environment ; 

transmittin g, by the mobile device, information identifying the user-selected job type 
mode to an inventory management system, in a single format which is consistent with the 
modality in which the first user input was received; 

responsive to transmitting the information, receivin g, by the mobile device, a response to 
the information identifying the user-selected job type mode from the inventory management 
system, in two or more formats which arc consistent with both the first modality and the second 
modality, the response identifying a location at which th e u se r warehouse worker will perform a 
function related to the user-selected job type mode on the one or more items, a description of the 
one or more items, and a best route from a present location of the warehouse worker to the 
particular storage location user to th e bin ; 

providin g, by the mobile device, the response to th e us e r the warehouse worker using 
both audio and display components of the multi-modal user interface, in the two or more formats 
which are consistent with both the first modality and the second modality; 
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receivin g, by the mobile device, a second user input using either the first modality or the 
second modality as chosen by the warehouse worker the us e r , the second user input indicating 
that the user warehouse worker has completed the selected function, and including count 
information or location information associated with the one or more items as observed by the 
warehouse worker th e user and, when the voice-based modality is used, a phonetically distinct 
checkword associat e d with th e bin to indicat e associated with and displayed at the particular 
storage location, the phonetically distinct checkword indicating completion of the selected 
function; 

updatin g, by the mobile device, inventory data for the one or more items based on the 
count information or the location information, the updated inventory data being accessible using 
the two or more formats which are consistent with both the first modality and the second 
modality; 

determining, by the mobile device, that the particular storage location is expected to be 

empty; 

responsive to determining that the particular storage location is expected to be empty, 
prompting, by the mobile device, the warehouse worker to confirm that the particular storage 
location is observed to actually be empty, using both the first modality and the second modality; 
and 

transmittin g, bv the mobile device, information indicating that th e us e r warehouse worker 
has completed the selected function related to the user-selected job type mode to the inventory 
management system, in a single format consistent with the modality in which the second user 
input was received. 

85. (Previously Presented) The method of claim 84, further comprising: 

outputting the updated inventory data in a format consistent with either the first modality 
or the second modality, such that the inventory data is maintained during performance of the 
inventory management tasks. 



86. (Previously Presented) The method of claim 84, wherein the job type mode comprises 
job data. 
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87. (Previously Presented) The method of claim 84, wherein the multi-modal interface 
further enables output and user input information using a Radio Frequency Identification Device 
(RF1D) modality, or a bar code modality. 

88. (Previously Presented) The method of claim 84, further comprising receiving a pick list 
identifying the one or more items and the function. 

89. (Previously Presented) The method of claim 88, further comprising: 
displaying the pick list using a visual user interface, and 

outputting, using voice synthesis, information identifying less than all items on the pick 



90. (Previously Presented) The method of claim 84, wherein the formats consistent with both 
the first modality and the second modality comprise HTML and VXML, respectively. 

91. -94. (Cancelled) 

95. (Previously Presented) The method of claim 84, wherein the first user input is received 
using a different modality than the second user input. 

96. (Previously Presented) The method of claim 95, wherein: 

the first user input is received using the voice-based modality, and 
the second user input is received using the touch-based modality. 



list. 



